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UNITED STATES DISTRICT COURT 
FOR THE DISTRICT OF COLUMBIA 


AMERICAN EDUCATIONAL RESEARCH ) 

ASSOCIATION, INC., AMERICAN ) 

PSYCHOLOGICAL ASSOCIATION, INC., ) 

and NATIONAL COUNCIL ON ) 

MEASUREMENT IN EDUCATION, INC., ) Civil Action No. 1:14-cv-00857-TSC-DAR 

) 

Plaintiffs, ) DECLARATION OF LAURESS 

) L. WISE IN SUPPORT OF 

v. ) PLAINTIFFS’ MOTION FOR 

) SUMMARY JUDGMENT AND ENTRY 
PUBLIC.RESOURCE.ORG, INC., ) OF A PERMANENT INJUNCTION 

) 

Defendant. ) 

_ ) 

I, LAURESS L. WISE, declare: 

1. I am the Immediate Past President of the National Council on Measurement in 
Education, Inc. (“NCME”). I have been a member of this organization for approximately 30 
years. I previously was the President of NCME from April 2014 through April 2015, and Vice 
President of this organization from April 2013 through April 2014. I submit this Declaration in 
support of the motion of the American Educational Research Association, Inc. (“AERA”), the 
American Psychological Association, Inc. (“APA”), and the NCME (collectively, “Plaintiffs” or 
“Sponsoring Organizations”) for summary judgment and the entry of a permanent injunction. 

2. I also am a principal scientist with the Human Resources Research Organization 
(“HumRRO”), spending full time on research and evaluation projects relating to educational 
measurement. I previously served as HumRRO CEO for 13 years, combining management and 
research activities and, before that, directed research and development for the Armed Services 
Vocational Aptitude Battery for the Department of Defense. Before that I spent 16 years as a 
researcher for the American Institutes for Research, rising to the position of Director of 
Research. I am also a member of both AERA and APA. 
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3. NCME is a District of Columbia not-for-profit corporation. 

4. NCME is a professional organization for individuals involved in assessment, 
evaluation, testing, and other aspects of educational measurement. NCME’s members are 
involved in the construction and use of standardized tests; new forms of assessment, including 
performance-based assessment; program design; and program evaluation. 

5. In 1955, AERA and NCME prepared and published a companion document to 
APA’s “Technical Recommendations for Psychological Tests and Diagnostic Techniques” 
(published in 1954), entitled “Technical Recommendations for Achievement Tests.” 

6. Subsequently, a joint committee of the three organizations modified, revised and 
consolidated the two documents into the first Joint Standards. Beginning with the 1966 revision, 
the Sponsoring Organizations collaborated in developing the “Joint Standards” (or simply, the 
“Standards”). Each subsequent revision of the Standards has been careful to note that it is a 
revision and update of that document. 

7. Beginning in the mid-1950s, the Sponsoring Organizations formed and 
periodically reconstituted a committee of experts in psychological and educational assessment, 
charged with the initial development of the Technical Recommendations and then each 
subsequent revision of the (renamed) Standards. These committees were formed by the three 
organizations’ Presidents (or their designees), who would meet and jointly agree on the 
membership. Often a chair or co-chairs of these committees were selected by joint agreement. 
Beginning with the 1966 version of the Standards, this committee became referred to as the 
“Joint Committee.” For example, I was the co-chair of the Joint Committee for the 2014 edition 
of the Standards. 

8. Financial and operational oversight for the Standards’ revisions, promotion, 
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distribution, and for the sale of the 1999 and 2014 Standards has been undertaken by a 
periodically reconstituted Management Committee, comprised of designees of the three 
Sponsoring Organizations. 

9. All members of the Joint Committee(s) and the Management Committee(s) are 
unpaid volunteers. The expenses associated with the ongoing development and publication of 
the Standards include travel and lodging expenses (for the Joint Committee and Management 
Committee members), support staff time, printing and shipment of bound volumes, and 
advertising costs. 

10. Many different fields of endeavor rely on assessments. The Sponsoring 
Organizations have ensured that the range of these fields of endeavor is represented in the Joint 
Committee’s membership - e.g., admissions, achievement, clinical counseling, educational, 
licensing-credentialing, employment, policy, and program evaluation. Similarly, the Joint 
Committee’s members represent expertise across major functional assessment areas - e.g., 
validity, equating, reliability, test development, scoring, reporting, interpretation, large scale 
interpolation and cognitive behavioral therapy. 

11. From the time of their initial creation to the present, the preparation and periodic 
revisions to the Standards entail intensive labor and considerable cross-disciplinary expertise. 
Each time the Standards are revised, the Sponsoring Organizations select and arrange for 
meetings of the leading authorities in psychological and educational assessments (known as the 
Joint Committee). During these meetings, certain Standards are combined, pared down, and/or 
augmented, others are deleted altogether, and some are created as whole new individual 
Standards. The 1999 version of the Standards is nearly 200 pages, and took more than five years 
to complete - resulting from work put in by the Joint Committee to generate a set of best 
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practices on educational and psychological testing that are respected and relied upon by leaders 
in their fields. 

12. The Standards originally were created as principles and guidelines - a set of best 
practices to improve professional practice in testing and assessment across multiple settings, 
including education and various areas of psychology. The Standards can and should be used as a 
recommended course of action in the sound and ethical development and use of tests, and also to 
evaluate the quality of tests and testing practices. Additionally, an essential component of 
responsible professional practice is maintaining technical competence. Many professional 
associations also have developed standards and principles of technical practice in assessment. 
The Sponsoring Organizations’ Standards have been and still are used for this purpose. 

13. The Standards, however, are not simply intended for members of the Sponsoring 
Organizations, AERA, APA, and NOME. The intended audience of the Standards is broad and 
cuts across audiences with varying backgrounds and different training. For example, the 
Standards also are intended to guide test developers, sponsors, publishers, and users by providing 
criteria for the evaluation of tests, testing practices, and the effects of test use. Test user 
standards refer to those standards that help test users decide how to choose certain tests, interpret 
scores, or make decisions based on tests results. Test users include clinical or industrial 
psychologists, research directors, school psychologists, counselors, employment supervisors, 
teachers, and various administrators who select or interpret tests for their organizations. There is 
no mechanism, however, to enforce compliance with the Standards on the part of the test 
developer or test user. The Standards, moreover, do not attempt to provide psychometric 
answers to policy or legal questions. 

14. The Standards promote the development of high quality tests and the sound use of 
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results from such tests. Without such high quality standards, tests might produce scores that are 
not defensible or accurate, not an adequate reflection of the characteristic they were intended to 
measure, and not fair to the person tested. Consequently, decisions about individuals made with 
such test scores would be no better, or even worse, than those made with no test score 
information at all. Thus, the Standards help to ensure that measures of student achievement are 
relevant, that admissions decisions are fair, that employment hiring and professional 
credentialing result in qualified individuals being selected, and patients with psychological needs 
are diagnosed properly and treated accordingly. Quality tests protect the public from harmful 
decision making and provide opportunities for education and employment that are fair to all who 
seek them. 

15. The Standards apply broadly to a wide range of standardized instruments and 
procedures that sample an individual’s behavior, including tests, assessments, inventories, scales, 
and other testing vehicles. The Standards apply equally to standardized multiple-choice tests, 
performance assessments (including tests comprised of only open-ended essays), and hands-on 
assessments or simulations. The main exceptions are that the Standards do not apply to 
unstandardized questionnaires ( e.g ., unstructured behavioral checklists or observational forms), 
teacher-made tests, and subjective decision processes (e.g., a teacher’s evaluation of students’ 
classroom participation over the course of a semester). 

16. The Standards have been used to develop testing guidelines for such activities as 
college admissions, personnel selection, test translations, test user qualifications, and computer- 
based testing. The Standards also have been widely cited to address technical, professional, and 
operational norms for all forms of assessments that are professionally developed and used in a 
variety of settings. The Standards additionally provide a valuable public service to state and 
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federal governments as they voluntarily choose to use them. For instance, each testing company, 
when submitting proposals for testing administration, instead of relying on a patchwork of local, 
or even individual and proprietary, testing design and implementation criteria, may rely instead 
on the Sponsoring Organizations’ Standards to afford the best guidance for testing and 
assessment practices. 

17. The Standards were not created or updated to serve as a legally binding document, 
in response to an expressed governmental or regulatory need, nor in response to any legislative 
action or judicial decision. However, the Standards have been cited in judicial decisions related 
to the proper use and evidence for assessment, as well as by state and federal legislators. These 
citations in judicial decisions and during legislative deliberations occurred without any lobbying 
by the Plaintiffs. 

18. NCME has never communicated with Congress for the purpose of encouraging 
the enactment of the Standards into law. 

19. Additionally, NCME has never solicited any government agency to incorporate 
the Standards into the Code of Federal Regulations or other rules of Federal or State agencies. 

20. In the policymaking arena, NCME believes the Standards should be treated as 
guidelines informing the enactment of legislation and regulations consistent with best practices 
in the development and use of tests - to insure that they are valid, reliable and fair. 

21. The Sponsoring Organizations promote and sell copies of the Standards via 
referrals to the AERA website, at annual meetings, in public offerings to students, and to 
educational institution faculty. Advertisements promoting the Standards have appeared in 
meeting brochures, in scholarly journals, and in the hallways at professional meetings. 
Accompanying this Declaration as Exhibit KKK is a true copy of advertisements for the 1999 
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Standards published in NCME’s Journal of Educational Management. These advertisements 
were produced at Bates Nos. AERA_APA_NCME_0031444-0031451. 

22. Distribution of the Standards is closely monitored by the Sponsoring 
Organizations. AERA, the designated publisher of the Standards, sometimes does provide 
promotional complementary print copies to students or professors. Except for these few 
complementary print copies, however, the Standards are not given away for free; and certainly 
they are not made available to the public by any of the three organizations for anyone to copy 
free of charge. 

23. To date, NCME has never posted, or authorized the posting of, a digitized copy of 
the 1999 Standards on any publicly accessible website. 

24. Without receiving at least some moderate income from the sales of the Standards 
to offset their production costs and to allow for further revisions, it is very likely that the 
Sponsoring Organizations would no longer undertake to periodically update them, and it is 
unknown who else would. 

25. In late 2013 and early 2014, the Sponsoring Organizations became aware that the 
1999 Standards had been posted on the Internet without their authorization, and that students 
were obtaining free copies from the posting source. Upon further investigation, the Sponsoring 
Organizations discovered that Public Resource was the source of the online posting. 

26. Public Resource posted Plaintiffs’ 1999 Standards to its website and the Internet 
Archive website without the permission or authorization of any of the Sponsoring Organizations. 

27. Plaintiffs have been made aware that at least some of those users who obtained 
the 1999 Standards for free from Public Resource did so to avoid paying the modest sale price 
for authorized print copies. 
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28. Accompanying this Declaration as Exhibit LLL is a true copy of an e-mail dated 
March 5, 2014 from Gregory J. Cizek to me regarding a student not purchasing the 1999 
Standards because “they [were] available for free online” at 
https://law.resource.org/pub/us/cfr/ibr/001/aera.standards.1999.pdf.” This e-mail exchange was 
marked as Exhibit 1252 during my deposition. 

I DECLARE, under the penalty of peijury, that the foregoing is true and correct. 


Dated: December &l . 2015 




Lauress L. Wise 



- 8 - 


GSebsb H I141-ffiw(300[SE57 : nS2D FRite±ll]2J2M/tE0 off2395 

EXHIBIT 1 

LAURESS L. WISE 

Curriculum Vitae 


OVERVIEW 


Dr. Lauress Wise has over 35 years’ experience in educational research and continues 
extensive work on educational policy and assessment issues. Dr. Wise currently advises 
several states and the PARCC assessment consortium on technical issues in test development 
and use. He serves on the Board of the National Council of Measurement in Education as the 
immediate past-president. He is also serving on a National Research Council Committee that is 
evaluating the NAEP achievement levels. He recently co-chaired the panel that revise the 
AERA/APA/NCME Standards for Educational and Psychological Testing and previously chaired 
the National Academy of Sciences Board on Testing and Assessment. Recent research and 
development efforts include a 15-year independent evaluation of the California High School Exit 
Exam and quality assurance work for the National Assessment of Educational Progress 
(NAEP). Dr. Wise previously served on several National Research Council committees, 
chairing the Committees on Scientific Research in Education and the Evaluation of the National 
Voluntary Tests. 


EDUCATION 


Ph.D, Mathematical Psychology 1975 University of California, Berkeley 

B.S., Mathematics, Psychology (with Distinction) 1967 Stanford University 


AREAS OF EXPERTISE 


Test Development and Validation 
Program and Policy Evaluation 
Test Use Policy 


Project Management 

Statistical and Psychometric Issues 

Computer-Based Testing 


PROFESSIONAL EXPERIENCE 


Human Resources Research Organization 1994 - 2015 

Principal Scientist 

• Served as HumRRO’s president from 1994 to 2007. Remained active in research on testing 
and test use policy. Directed two major HumRRO educational testing projects, one to 
provide quality assurance for the National Assessment of Educational Progress (NAEP) and 
the other an independent evaluation of California’s High School Exit Exam (CAHSEE). He 
continues to serve as a senior psychometric advisor for a graduate school admissions 
testing program. 

• Served as co-chair of the committee that revised the 1999 AERA/APA/NCME Standards for 
Educational and Psychological Testing, and has previously served as Chair of the National 
Academy of Science (NAS) Board on Testing and Assessment and chaired the NAS 
Committee on Research in Education. 

• Currently serves on technical advisory committees for the Hawaii, Wyoming, Utah, 
Tennessee, and Virginia departments of education, and the Partnership for Assessing 
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Readiness for College and Career (PARCC) advisory committees. Also serves on the 
Rhode Island Technical Advisory Committee for Teacher Evaluation. 

• Served as co-Principal Investigator on the first year of the Congressionally-mandated 
evaluation of President Clinton’s Voluntary National Tests and chaired the NAS committee 
that performed the second year of that evaluation, and on the NAS committee to evaluate 
the NAEP and on the National Academy of Education's Panel for the Evaluation of the 
NAEP Trial State NAEP. 

• Other work includes vertical alignment of state content standards, modeling the effects of 
motivation on examinee performance on low-stakes assessments, the impact of changes in 
exclusions on NAEP results for Kentucky, and scaling constructed response and multiple 
choice items on the Florida assessment. Dr. Wise also worked on the development and 
validation of a computer-administered assessment now used for selection of air traffic 
controllers and developed a computer-based system for assessing work values as part of a 
Department of Labor effort to develop improved career guidance tools. 

Defense Manpower Data Center 1990 - 1994 

Chief, Personnel Testing Division 

• Spokesperson for the Department of Defense on matters relating to the development and 
use of cognitive tests. Dr. Wise’s unit was responsible for all research and development for 
the Armed Services Vocational Aptitude Battery (ASVAB). 

• Work included evaluation and implementation of a computerized, adaptive version of the 
ASVAB, automated item and form development procedures, new career exploration 
procedures for use with the high school testing program, the development and testing of a 
new career interest inventory, and extensive validity research. 

American Institutes for Research (AIR) 1974 - 1990 

Associate Research Scientist to Director of Research 

• Directed a variety of studies and projects, including the Review and Analysis of the General 
Aptitude Test Battery (GATB) Project for the U.S. Employment Service within the 
Department of Labor, the Army Synthetic Validation Project, the Army's Computerized 
Adaptive Screening Test (CAST) Revision Project, and validation studies for the Software 
Engineering Institute at Carnegie Mellon University. 

• Served as director of analysis for the Army's massive Project A, analyzing new selection 
tests, developing models of performance in a variety of jobs, and assessing the validity of 
each new test for predicting different facets of performance in different jobs. 

• Served for twelve years as the chief psychometrician for the Medical College Admissions Test, 
developing procedures for the screening and calibration of new items and for the construction 
and equating of new forms. Also consulted with the Department of Education on issues related 
to testing and data analysis as part of the Statistical Analysis Group in Education. 

• From 1978 to 1982, served as Director of Project TALENT, a nationally representative 
longitudinal study of nearly 400,000 members of the high school classes of 1960 through 
1963. Oversaw the collection of the final wave of follow-up data and conducted targeted 
research on issues such as gender differences in mathematics achievement, school 
differences in student achievement, the development of careers in science and medicine, 
and the consequences of adolescent childbearing. 
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University of California 

Programmer and Instructor 

• While a graduate student at the University of California, Dr. Wise served as the computer 
consultant for the Psychology Department, helping both faculty and students in the design and 
execution of data analyses. He also taught an undergraduate course in Psychological Statistics. 

California Department of Public Health 1968 - 1972 

Computer Programmer and Data Processing Systems Analyst 

• Created data systems to support licensing functions and vital statistics systems at the 
Department of Public Health. Was in-house project manager for a new management 
information system that involved defining "outputs" for each bureau and department and 
relating these outputs to costs. 


PROFESSIONAL AFFILIATIONS AND SERVICE 


• American Educational Research Association (AERA) 

• American Psychological Association (APA) 

Divisions 5, 14, 19 

• Member, National Council on Measurement in Education (NCME) 

• Psychometric Society 

SELECTED BIBLIOGRAPHY 


Publications 

Wise, L. L. (in press). How we got to where we are: Evolving policy demands for the next 
generation assessments. In Next Generation Assessments. R. Lissitz, Ed. (To be published in 
2016. 
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Handbook of Test Development. New York, NY: Routledge. 
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Educational Measurement: Issues and Practice, 33(4), 4-12. 
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and Conclusions. Applied Measurement in Education, 23(2), 209-214. 
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Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The 
National Academy Press. 
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Inquiry to Operation. Washington, DC: American Psychological Association. 

Wolfe, J.H., Alderton, D.L., Larson, G.E., Bloxom, B., & Wise, L.L. (1997). Expanding the 
content of CAT-ASVAB: New tests and their validity. In W.A. Sands, B.K. Waters, & J.R. 
McBride (Eds.), Computerized Adaptive Testing: From Inquiry to Operation. Washington, DC: 
American Psychological Association. 
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